‘Checkers’ - Highly Efficient Derivative-Free Optimization

Derivative-free algorithms are frequently required for the optimization of nonsmooth functions defined by physical experiments or by averaging of the statistics of numerical simulations of chaotic systems such as turbulent flows. The core idea of all efficient algorithms for problems of this class is to keep function evaluations far apart until convergence is approached. Generalized Pattern Search (GPS) algorithms, such as the present, accomplish this by coordinating the search with an underlying grid which is refined and coarsened as appropriate. Rather than using the Cartesian grid (the typical choice), the present work introduces for this purpose the use of lattices derived from n-dimensional sphere packings (for a comprehensive review of such lattices and their properties see Conway & Sloane 1999). Such lattices are significantly more uniform and have much higher kissing numbers (that is, they have many more nearest neighbors) than their Cartesian counterparts; both of these facts make them much better suited for coordinating GPS algorithms. One of the most efficient subclasses of GPS algorithms, known as the Surrogate Management Framework (SMF; see Booker et al, 1999), alternates between an exploratory Search over a surrogate function interpolating all existing function evaluations (and thus summarizing the trends which they represent), and an exhaustive Poll which checks the function on neighboring points to confirm or confute the local optimality of any given Candidate Minimum Point (CMP) on the underlying grid. The present work combines the SMF with efficient lattices based on n-dimensional sphere packings, and additionally incorporates one of the highly efficient global search strategies meticulously examined by Jones (2001), thereby developing an extremely efficient lattice-based derivative-free optimization algorithm. Our code implementing this algorithm, dubbed Checkers, compares quite favorably to competing algorithms on a range of well-known optimization test problems.